Information processing apparatus, information processing method, and program

ABSTRACT

In one example embodiment, an information processing apparatus, for an observed image associated with an observation target object (e.g., a section of biological tissue), associates and stores position information and observation magnification information. In this embodiment, the information processing apparatus causes a display device to: (i) display an image associated with the observation target object; (ii) indicate the first positional information of the first observed image; and (iii) indicate the first observation magnification information of the first observed image.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application is a divisional of U.S. application Ser. No.12/900,000, filed Oct. 7, 2010, which claims priority to Japanese PatentApplication No. JP 2009-269495, filed in the Japanese Patent Office onNov. 27, 2009, the entire content of which is being incorporated hereinby reference.

BACKGROUND

In a field of medicine, pathology, or the like, there has been proposeda system that digitizes an image of a cell, a tissue, an organ, or thelike of a living body, that is obtained by an optical microscope, toexamine the tissue or the like by a doctor or a pathologist or diagnosea patient based on the digitized image.

For example, Japanese Patent Application Laid-open No. 2009-37250(hereinafter, referred to as Patent Document 1) discloses a method inwhich an image optically obtained by a microscope is digitized by avideo camera with a CCD (charge coupled device), a digital signal isinput to a control computer system, and the image is visualized on amonitor. A pathologist performs examination while watching the imagedisplayed on the monitor (see, for example, paragraphs 0027 and 0028 andFIG. 5 of Patent Document 1).

Generally, as a magnification of a microscope is increased, anobservation area thereof becomes smaller relative to an entireobservation target. For example, a pathologist often scans and observesthe entire observation target by a microscope and observes a part of theentire observation target at a particularly high magnification toexamine the target. In such an examination, if there is a disorder in anarea where the pathologist does not observe in the observation target,that is, a pathologist misses a disorder, a significant problem mayarise later.

In view of the above-mentioned circumstances, it is desirable to providean information processing apparatus, an information processing method,and a program for avoiding a risk of missing in the observation targetwith a microscope by a user.

It is also desirable to provide an information processing apparatus, aninformation processing method, and a program that are useful foreducation in fields in which such an observation target is treated.

SUMMARY

The present disclosure relates to an information processing apparatus,an information processing method, and a program for controlling displayof an image obtained by a microscope in a field of medicine, pathology,biology, materials science, or the like.

In an example embodiment, the information processing apparatus includesa processor and a memory device operatively coupled to the processor,the memory device storing instructions that cause the processor, incooperation with the memory device, to: (a) for a first observed imageassociated with an observation target object (e.g., a section ofbiological tissue), associate and store first position information andfirst observation magnification information; and (b) cause a displaydevice to: (i) display an image associated with the observation targetobject; (ii) indicate the first positional information of the firstobserved image; and (iii) indicate the first observation magnificationinformation of the first observed image. In an example embodiment, thefirst observed image is observed by a microscope.

In an example embodiment, the displayed image associated with theobservation target object includes an entire image of the observationtarget object. In this example, the instructions cause the displaydevice to display the first observed image such that the first observedimage is included in the displayed entire image of the observationtarget object.

In an example embodiment, the first observed image is from a pluralityof different images associated with the observation target object. In anexample embodiment, the plurality of different images form an imagepyramid structure.

In an example embodiment, the displayed image has one of a firstresolution and a second resolution.

In an example embodiment, each of the indicated first positionalinformation and the indicated first observation magnificationinformation overlap the displayed image.

In an example embodiment, the instructions cause the display device toindicate the first positional information by indicating positionalinformation of the entire first observed image, wherein the indicationof the entire first observed image overlaps the displayed image.

In an example embodiment, the instructions, cause the processor to, inresponse to a request for a change in observance from the first observedimage to a second image to be observed, associate and store the firstposition information and the first observation magnificationinformation.

In an example embodiment, the instructions cause the processor to causethe display device to indicate the first positional information using animage (e.g., an arrow image).

In an example embodiment, the image used to indicate the firstpositional information has a color based on the first observationmagnification information.

In an example embodiment, the image used to indicate the firstpositional information has a size based on the first observationmagnification information.

In an example embodiment, the instructions cause the processor to causethe display device to indicate the first positional information and thefirst observation magnification information using outline images whichindicate an entire outline wherein a combination of individuallyobserved images are combined.

In an example embodiment, the instructions cause the processor to causethe display device to indicate an amount of time the first observedimage was observed.

In an example embodiment, the instructions cause the processor to causethe display device to indicate user identification informationassociated with the first observed image.

In an example embodiment, the instructions cause the processor to: (a)for a second observed image associated with the observation targetobject, associate and store second position information and secondobservation magnification information; (b) indicate the secondpositional information of the second observed image; and (c) indicatethe second observation magnification information of the second observedimage.

In an example embodiment, the instructions cause the processor to causethe display device to: (a) indicate the first positional informationusing a first image; and (b) indicate the second positional informationusing a second image such that a temporal order of the first observedimage and the second observed image is indicated.

In an example embodiment, the instructions cause the processor toassociate and store first position information and first observationmagnification information based on a predetermined sampling period.

In an example embodiment, the method of operating the informationprocessing apparatus includes: (a) causing a processor to execute theinstructions to, for a first observed image associated with anobservation target object, associate and store first positioninformation and first observation magnification information; and (b)causing the processor to execute the instructions to cause a displaydevice to: (i) display an image associated with the observation targetobject; (ii) indicate the first positional information of the firstobserved image; and (iii) indicate the first observation magnificationinformation of the first observed image.

In an example embodiment, the computer-readable medium storinginstructions causes an information processing apparatus to: (a) for afirst observed image associated with an observation target object,associate and store first position information and first observationmagnification information; and (b) cause a display device to: (i)display an image associated with the observation target object; (ii)indicate the first positional information of the first observed image;and (iii) indicate the first observation magnification information ofthe first observed image.

In an example embodiment, the information processing apparatus includesa processor and a memory device operatively coupled to the processor,the memory device storing instructions that cause the processor, incooperation with the memory device, to, for a first observed imageassociated with an observation target object (e.g., a section ofbiological tissue), associate and store first position information andfirst observation magnification information.

In an example embodiment, the instructions cause the processor to, inresponse to a request for a change in observance from the first observedimage to a second image to be observed, associate and store the firstposition information and the first observation magnificationinformation.

In an example embodiment, the instructions, when executed by theprocessor, cause the processor to associate and store first positioninformation and first observation magnification information based on apredetermined sampling period.

In an example embodiment, the method of operating an informationprocessing apparatus includes causing a processor to execute theinstructions to, for a first observed image associated with anobservation target object, associate and store first positioninformation and first observation magnification information.

In an example embodiment, the computer-readable medium storesinstructions structured to cause an information processing apparatus to,for a first observed image associated with an observation target object,associate and store first position information and first observationmagnification information.

In an example embodiment, the information processing apparatus includesa processor and a memory device operatively coupled to the processor,the memory device storing instructions that cause the processor, incooperation with the memory device, to cause a display device to: (a)display a first image associated with an observation target object(e.g., a section of biological tissue); (b) indicate first positionalinformation of a first observed image; and (c) indicate firstobservation magnification information of the first observed image.

In an example embodiment, each of the indicated first positionalinformation and the indicated first observation magnificationinformation overlap said displayed first image.

In an example embodiment, the instructions cause the processor to causethe display device to indicate the first positional information byindicating positional information of the entire first observed image,said indication of the entire first observed image overlapping saiddisplayed first image.

In an example embodiment, the instructions cause the processor to, inresponse to a request for a change in observance from the first observedimage to a second image to be observed, associate and store the firstposition information and the first observation magnificationinformation.

In an example embodiment, the instructions cause the processor to causethe display device to indicate the first positional information using asecond image (e.g., an arrow image). In an example embodiment, thesecond image used to indicate the first positional information has acolor based on the first observation magnification information. In anexample embodiment, the second image used to indicate the firstpositional information has a size based on the first observationmagnification information.

In an example embodiment, the instructions cause the processor to causethe display device to indicate the first positional information and thefirst observation magnification information using outline images whichindicate an entire outline wherein a combination of individuallyobserved images are combined.

In an example embodiment, the instructions cause the processor to causethe display device to indicate an amount of time the first observedimage was observed.

In an example embodiment, the instructions cause the processor to causethe display device to indicate user identification informationassociated with the first observed image.

In an example embodiment, the instructions, when executed by theprocessor, cause the processor to: (a) indicate second positionalinformation of a second observed image; and (b) indicate secondobservation magnification information of a second observed image.

In an example embodiment, the instructions cause the processor to causethe display device to: (a) indicate the first positional informationusing a second image; and (b) indicate the second positional informationusing a third image such that a temporal order of the first observedimage and the second observed image is indicated.

In an example embodiment, the instructions cause the processor toassociate and store first position information and first observationmagnification information based on a predetermined sampling period.

In an example embodiment, the method of operating an informationprocessing apparatus includes: (a) causing a processor to execute theinstructions to cause a display device to display a first imageassociated with an observation target object; (b) causing the processorto execute the instructions to cause the display device to indicatefirst positional information of a first observed image; and (c) causingthe processor to execute the instructions to cause the display device toindicate first observation magnification information of the firstobserved image.

In an example embodiment, the computer-readable medium storesinstructions structured to cause an information processing apparatus to:(a) display a first image associated with an observation target object;(b) indicate first positional information of a first observed image; and(c) indicate first observation magnification information of the firstobserved image.

In an example embodiment, the information processing apparatus includesa processor and a memory device operatively coupled to the processor,the memory device storing instructions that cause the processor, incooperation with the memory device, to: (a) for a first image associatedwith a section of biological tissue which is observed by a microscope,associate and store first position information; and (b) cause a displaydevice to: (i) display an image associated with the section ofbiological tissue; and (ii) indicate the first positional information ofthe first observed image.

As described above, it is possible to avoid the risk of missing theobservation target using the microscope by the user.

These and other objects, features and advantages of the presentdisclosure will become more apparent in light of the following detaileddescription of best mode embodiments thereof, as illustrated in theaccompanying drawings.

Additional features and advantages are described herein, and will beapparent from the following Detailed Description and the figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing the structure of an informationprocessing system including at least an information processing apparatusaccording to a first embodiment of the present disclosure;

FIG. 2 is a diagram showing an image pyramid structure for explaining adisplay principle thereof;

FIG. 3 is a diagram for explaining a procedure at a time when an imagegroup of the image pyramid structure is generated;

FIG. 4 is a flowchart showing the processing of a PC that is anoperation of information processing according to the first embodiment;

FIG. 5 is a diagram showing an entire image including an image of anobservation target object for explaining the operation;

FIGS. 6A, 6B, and 6C are diagrams each showing an example of a partialimage;

FIG. 7 is a lookup table of history information of the partial imagestored in a storage unit in FIG. 4;

FIG. 8 is a diagram showing an example of an entire image at anarbitrary resolution in which a history image is composed;

FIG. 9 is a diagram showing an entire image at an arbitrary resolutionaccording to a second embodiment of the present disclosure;

FIG. 10 is a diagram showing an entire image at an arbitrary resolutionaccording to a second embodiment of the present disclosure;

FIG. 11 is a diagram showing an entire image at an arbitrary resolutionaccording to a third embodiment of the present disclosure; and

FIG. 12 is a diagram showing an entire image at an arbitrary resolutionaccording to a fourth embodiment of the present disclosure.

DETAILED DESCRIPTION

Hereinafter, embodiments of the present disclosure will be describedwith reference to the drawings.

First Embodiment Structure of Information Processing Apparatus

FIG. 1 is a block diagram showing the structure of an informationprocessing system including at least an information processing apparatusaccording to an embodiment of the present disclosure. As the informationprocessing apparatus, a PC (personal computer) 100 is used, for example.

The PC 100 includes a CPU (central processing unit) 101, a ROM (readonly memory) 102, a RAM (random access memory) 103, an input and outputinterface (hereinafter, abbreviated as I/O interface) 105, and a bus 104that connects those components with one another.

To the I/O interface 105, a display unit 106, an input unit 107, astorage unit 108, a communication unit 109, a drive unit 110, and thelike are connected.

The display unit 106 is a display device that uses liquid crystal, EL(electro-luminescence), a CRT (cathode ray tube), or the like.

The input unit 107 is, for example, a pointing device, a keyboard, atouch panel, or another operation apparatus. In the case where the inputunit 107 includes a touch panel, the touch panel may be integrated withthe display unit 106.\

The storage unit 108 is a non-volatile storage device such as an HDD(hard disk drive), a flash memory, and another solid-state memory.

The drive unit 110 is a device capable of driving a removable recordingmedium 111 such as an optical recording medium, a floppy (registeredtrademark) disk, a magnetic recording tape, and a flash memory. Incontrast, the storage unit 108 is often used as a device that ispreviously included in the PC 100 and mainly drives a recording mediumthat is not removable.

The communication unit 109 is a modem, a router, or anothercommunication apparatus that is connectable to a LAN (local areanetwork), a WAN (wide area network), or the like and is used forcommunicating with another device. The communication unit 109 mayperform either one of a wired communication or a wireless communication.The communication unit 109 is used separately from the PC 100 in manycases.

Next, a description will be given on an image that is obtained by anoptical microscope (not shown) and is mainly stored in the storage unit108 of the PC 100 and on a principle of displaying the image. FIG. 2 isa diagram showing an image pyramid structure for explaining the displayprinciple.

An image pyramid structure 50 in this embodiment is an image group(entire image group) generated at a plurality of resolutions withrespect to one image obtained from one observation target object 15(see, FIG. 3) by the optical microscope. On a lowermost part of theimage pyramid structure 50, a largest image is disposed, and on anuppermost part thereof, a smallest image is disposed. A resolution ofthe largest image is 50×50 (Kpixel: kilopixel) or 40×60 (Kpixel), forexample. A resolution of the smallest image is 256×256 (pixel) or256×512 (pixel), for example.

That is, when the display unit 106 displays those images at the samemagnification (100%) (displays each image by the number of dots that isphysically the same as the number of pixels of the images), the largestimage is displayed in the largest size, and the smallest image isdisplayed in the smallest size. Here, a display range of the displayunit 106 is represented by D in FIG. 2.

FIG. 3 is a diagram for explaining a procedure at a time when the imagegroup of the image pyramid structure 50 is generated.

First, a digital image of an original image obtained at a predeterminedobservation magnification by an optical microscope (not shown) isprepared. The original image corresponds to the largest image that isthe lowermost image of the image pyramid structure 50 shown in FIG. 2,that is, an image at a highest resolution. Therefore, as the lowermostimage of the image pyramid structure 50, an image obtained by beingobserved at a relatively high magnification by the optical microscope isused.

It should be noted that in the field of pathology, generally, a matterobtained by slicing an organ, a tissue, or a cell of a living body, or apart thereof is an observation target object 15. Then, a scannerapparatus (not shown) having a function of the optical microscope readsthe observation target object 15 stored on a glass slide, to obtain adigital image and store the digital image obtained into the scannerapparatus or another storage apparatus.

As shown in FIG. 3, the scanner apparatus or a general-purpose computer(not shown) generates, from the largest image obtained as describedabove, a plurality of images whose resolutions are reduced stepwise, andstores those images in unit of “tile” that is a unit of a predeterminedsize, for example. The size of one tile is 256×256 (pixel), for example.The image group generated as described above forms the image pyramidstructure 50, and the storage unit 108 of the PC 100 stores the imagepyramid structure 50. Actually, the PC 100 only has to store the imageswhose resolutions are different with the images being associated withresolution information items, respectively. It should be noted that thegenerating and storing the image pyramid structure 50 may be performedby the PC 100 shown in FIG. 1.

The entire image group that forms the image pyramid structure 50 may begenerated by a known compression method. For example, a knowncompression method used at a time when a thumbnail image is generatedmay be used.

The PC 100 uses software that employs the system of the image pyramidstructure 50, to extract a desired image from the image pyramidstructure 50 and output the desired image to the display unit 106 inaccordance with an input operation through the input unit 107 by theuser. Specifically, the PC 100 displays an image of an arbitrary partselected by the user, out of the images at an arbitrary resolutionselected by the user. With this operation, the user can get a feeling ofobserving the observation target object 15 while changing theobservation magnification. That is, the PC 100 functions as a virtualmicroscope. A virtual observation magnification in this case correspondsto a resolution in reality.

Operation of Information Processing Apparatus

FIG. 4 is a flowchart showing the processing of the PC 100, that isinformation processing according to this embodiment. FIG. 5 is a diagramshowing the entire image including an image of an observation targetobject for explaining the operation thereof. The entire image is animage at an arbitrary resolution, out of the image group (entire imagegroup) that forms the image pyramid structure 50.

The following processing of the PC 100 is implemented while softwarestored in the storage unit 108, the ROM 102, or the like and a hardwareresource of the PC 100 are cooperated with each other. Specifically, theCPU 101 loads a program that forms the software stored in the storageunit 108, the ROM 102, or the like and executes the program, therebyimplementing the following processing.

The user accesses a file including the image group of the image pyramidstructure 50 by an input operation using the input unit 107. Inaccordance with the operation, the CPU 101 of the PC 100 extracts apredetermined partial image from the image pyramid structure 50 storedin the storage unit, and causes the display unit 106 to display theimage extracted (Step 101). Out of the image pyramid structure 50, thepredetermined partial image accessed by the CPU 100 first may be set asappropriate by default or by the user.

The partial image refers to a part of the image at an arbitraryresolution out of the entire images stored and generated for eachresolution as shown in FIGS. 2 and 3. The partial image corresponds tothe image in the display range D displayed by the display unit 106.Here, the display range D does not indicate the size of the maximumdisplay range of the display unit 106, but indicates the entire or apart of the display range of the display unit 106. The display range Dcan be set as appropriate by a user setting, for example. FIGS. 6A to 6Care diagrams each showing the partial image.

In Step 101, the CPU 101 displays a partial image in the entire imagehaving a resolution corresponding to a relatively low resolution (lowmagnification) in general, for example, an observation magnification of1.25 times first.

In Step 102, the CPU 101 is in a standby state of an input operationfrom the input unit 107 by the user.

When the user operates the input unit 107 to change the display range Dto a desired range, the CPU 101 causes a partial image correspondingthereto to be displayed (Step 103). As shown in FIG. 5, for example, theuser shifts and changes a display range D1 to a display range D2 orscales up the observation magnification from the display range D2,thereby obtaining a display range D3 scaled up (display range isreduced). During this operation, the CPU 101 associates positionalinformation items of partial images that are output with informationitems on resolutions as observation magnifications for each partialimage output. The CPU 101 stores the associated information in the RAM103 or the storage unit 108 as history information of the partial images(Step 104).

In Steps 101 to 103, the CPU 101, the I/O interface 105, and the likefunction as output means for outputting the partial image.

A description will be given on an example of a processing in more detailin Steps 103 and 104.

FIG. 7 is a diagram showing a lookup table of the history information ofthe partial images stored in Step 104.

The assumption is made that the user operates the input unit 107,typically, drags a mouse to shift and change the display range in oneentire image from the display range D1 to the display range D2 differenttherefrom, as shown in FIG. 5. During this operation, the CPU 101stores, in the RAM 103 or the storage unit 108, the positionalinformation of the partial images corresponding to the display ranges D1and D2 and positional information of partial images on a way from thedisplay range D1 to the display range D2. In addition, the resolutioninformation of the partial images on the way from the display range D1to the display range D2 is stored with the resolution information beingassociated with the partial images. In this example, on the way from thedisplay range D1 to the display range D2, the resolution is not changed,and therefore the same resolution information is stored. In the exampleshown in FIG. 7, time when sampling is performed is also stored.

The storing processing of history information 20 as described above isperformed with a predetermined sampling period. That is, the CPU 101stores the positional information of all the partial imagescorresponding to the display range D displayed on the display unit 106in the one entire image with the predetermined sampling period. Thesampling period is 0.1 to 10 seconds, for example, but is not limited tothis range.

After the partial image of the display range D2 is output, if the inputoperation is not performed by the user for a predetermined time period,the CPU 101 may temporarily stop the storing processing of the historyinformation 20.

The positional information of the partial image refers to positionalinformation in the entire image and is managed as coordinate information(x, y), for example. Typically, the coordinate information of a centerposition of the partial image is managed. However, the coordinateinformation to be managed is not limited to that of the center position.

The assumption is made that the user shifts the display range from thedisplay range D1 to the display range D2, and then operates the inputunit 107 to change the observation magnification from 1.25 times to 20times. At this time, the CPU 101 changes the display range D from thedisplay range D2 to a display range D3 that is smaller than the displayrange D2. That is, the CPU 101 displays, out of the entire image at ahigher resolution than that of the partial images displayed in thedisplay ranges D1 and D2, a partial image having coordinate informationcorresponding to the coordinate information of the partial image of thedisplay range D2 as the display range D3.

Depending on the sampling period, during the shift of the observationmagnification from 1.25 times to 20 times, the CPU 101 also storesstepwise resolution information and positional information of thepartial images during the shift. Alternatively, in the case where therelative position of the partial image in the entire image is notchanged, but only the resolution information is changed through theoperation by the user, the CPU 101 may omit the storing processing ofthe stepwise history information (in this case, at least one of thepositional information and the resolution information) during the shiftor may delete the history information at a predetermined timingthereafter if stored.

Next, the user shifts the display range from the display range D3 to adisplay range D4, shifts the display range D4 to a display range D5, themagnification of which is higher than that of the display range D4, forexample, 40 times, and shifts the display range D5 to a display rangeD6. During those shifts, the CPU 101 stores the history information 20of the partial images as shown in FIG. 7. In the example shown in FIG.7, subsequently, the display range is shifted to the display ranges D7,D8, . . . .

As described above, FIGS. 6A to 6C are diagrams each showing an exampleof the partial image.

FIG. 6A is a diagram showing the display range D1 shown in FIG. 5 thatis a partial image at the observation magnification of 1.25 times.

FIG. 6B is a diagram showing the display range D3 shown in FIG. 5 thatis a partial image at the observation magnification of 20 times.

FIG. 6C is a diagram showing the display range D5 shown in FIG. 5 thatis a partial image at the observation magnification of 40 times.

In Step 104, the CPU 101, the storage unit 108, and the like function asthe storing means for storing the history information 20.

An input operation is performed for closing a file to terminate theobservation of the observation target object 15 by the user (YES in Step105). Then, the CPU 101 generates, based on the history information 20stored, a history image that represents traces of the partial images andthe observation magnifications in the entire image at an arbitraryresolution (Step 107). At this time, at least the CPU 101 functions asan image generation means. Step 107 will be described later.

In the case where the user does not terminate the observation of theobservation target object 15, the CPU 101 performs the same processingas Step 102 (Step 106).

The entire image at the arbitrary resolution in which the history imageis generated is an arbitrary entire image (including the original image)among the entire image group of the image pyramid structure 50.

Alternatively, the entire image at the arbitrary resolution may be anentire image at an arbitrary resolution formed when necessary forgeneration of the history image from at least one arbitrary image amongthe entire image group of the image pyramid structure 50 previouslyformed. For example, the CPU 101 can generate an entire image at aresolution corresponding to the observation magnification of 30 times,from the entire images of the observation magnifications of 20 times and40 times which are previously generated and stored by interpolation.

The entire image at the arbitrary resolution generated as describedabove may be used as a thumbnail image on the PC 100 or anothercomputer. That is, the CPU 101 may store, out of the entire image groupof the image pyramid structure 50, at least one entire image among theentire images other than the original image in the storage unit 108 asthe thumbnail image. The history image may be generated in the thumbnailimage.

Alternatively, the history image may be generated in the entire image ona screen which is to be displayed on the display unit 106 by the PC 100.

As a method of composing the history image into the entire image, abitmap composition or other known methods can be used.

The timing at which the thumbnail image is generated may be a timing atwhich the file of the entire image group that forms the image pyramidstructure 50 is stored in the storage unit 108 or at which the useraccesses the stored file for the first time.

FIG. 8 is a diagram showing examples of the entire image (including theoriginal image) at the arbitrary resolution generated as describedabove, the thumbnail image generated as described above, or the entireimage on the screen which is to be displayed on the display unit 106 bythe PC 100 (hereinafter, referred to as an entire image at an arbitraryresolution). In Step 107, specifically, the CPU 101 composes, in anentire image 40 at an arbitrary resolution, arrow images A forindicating the positional information of the partial images for eachsampling and the traces of the display ranges D corresponding to theresolution information. In FIG. 8, the arrow images A whose colors aredifferent depending on observation magnifications are generated. Forexample, arrow images A1 and A4 indicate the observation magnificationof 1.25 times. Arrow images of A2 and A5 indicate the observationmagnification of 20 times. Arrow images of A3 and the like indicate theobservation magnification of 40 times.

As described above, in this embodiment, based on the history information20 as information obtained by associating the positional information andthe resolution information of the plurality of partial images, the arrowimages A are generated in the entire image 40 at the arbitraryresolution as the history images that indicate the traces and theobservation magnifications. As a result, the user can grasp anobservation history by referring to the entire image 40 at the arbitraryresolution. Therefore, it is possible to avoid a risk of missing theobservation target by the microscope, which is useful particularly forthe field of medicine or pathology.

Further, in addition to the grasping of the observation history ofhim/herself by the user, another user can refer to the observationhistory. Thus, the observation history is highly useful for checking ofthe observation history of another person or for education.

In addition, in this embodiment, the colors of the arrow images A aredifferent depending on the observation magnifications. Therefore, it ispossible to grasp the observation magnification of the past observationby a person concerned or another person, which increases theconvenience.

In this embodiment, the traces of the partial images are indicated bythe arrow images A in particular, and therefore the arrow images Afunction as order emphasizing images. As a result, it is possible tointuitively grasp the temporal order of the past observations by theperson concerned or another person.

The order emphasizing images are not limited to the arrow images A, aslong as two partial images are connected with a straight-line orcurved-line image, for example.

The timing at which the history image is generated by the CPU 101 is notlimited to the timing in Step 107 in FIG. 4. For example, the CPU 101may generate the history image each time at least Step 103 is executed.

In the case where the history information is stored in the RAM 103 inStep 104, when the user closes the file to terminate the observation ofthe observation target object 15, the CPU 101 stores the historyinformation in the storage unit 108. Of course, even in the case wherethe history information is stored in the RAM 103, the CPU 101 mayperiodically store the history information in the storage unit 108.

Second Embodiment

FIG. 9 is a diagram showing an entire image at an arbitrary resolutionaccording to another embodiment of the present disclosure. Entire imagesat arbitrary resolutions according to the second to fifth embodimentsdescribed below are also generated by executing the process shown inFIG. 4 by the CPU 101.

In this embodiment, frame-like images F (F1, F2, F3, . . . ) arecomposed in an entire image 60 at an arbitrary resolution. Theframe-like images F (hereinafter, referred to as frame image F) functionas display-range emphasizing images for indicating the display ranges ofthe respective partial images. Further, frame images F1 and F2 are thedisplay ranges corresponding to the observation magnification of 1.25times, and frame images F3 and F4 are the display ranges correspondingto the observation magnification of 20 times. Further, frame images F5and F6 are the display ranges corresponding to the observationmagnification of 40 times. The sizes of the frame images F differdepending on the observation magnifications, and therefore the user canintuitively grasp the observation magnification.

In addition, in this embodiment, the frame images F whose colors aredifferent depending on the observation magnifications are generated,thereby making the magnification discrimination easier. As the imagesthat indicate the observation magnifications, images whose frames areindicated with various kinds of lines (that are different in width orare indicated with a solid line or a broken line, for example) may beused instead of the images having different colors.

The shape of the frame images F is not limited to the rectangular shape,and a shape in accordance with the shape displayed on the display unit106 may be used.

Third Embodiment

FIG. 10 is a diagram showing an entire image at an arbitrary resolutionaccording to another embodiment of the present disclosure.

In this embodiment, history images including outline images R (R1, R2,R3, . . . ) are composed in an entire image 70 at an arbitraryresolution. The outline images R indicate an entire outline in whichdisplay ranges of partial images for each observation magnification inthe entire image are combined. Further, the arrow images A are alsogenerated. The outline images R also function as the display-rangeemphasizing images. In addition, the colors of the outline images Rdiffer depending on the observation magnifications. Therefore, the usercan intuitively grasp the observation magnification.

As described above, the whole of the outline images R in which thedisplay ranges are combined is generated, with the result that the usercan reliably avoid the risk of missing the observation target object.

Fourth Embodiment

FIG. 11 is a diagram showing an entire image at an arbitrary resolutionaccording to another embodiment of the present disclosure.

An entire image 80 at an arbitrary resolution according to thisembodiment is obtained by further composing time-stamp images S (S1, S2,S3, . . . ) in the entire image 70 at the arbitrary resolution shown inFIG. 10. The time-stamp images S function as the order emphasizingimages. As a result, the user can easily grasp the time course of thetraces of the display ranges. The time-stamp images S may be generatedbased on time information of the sampling out of the history information20 shown in FIG. 7.

Fifth Embodiment

FIG. 12 is a diagram showing an entire image at an arbitrary resolutionaccording to another embodiment of the present disclosure.

In the entire image 90 at an arbitrary resolution according to thisembodiment, user identification images T (T1 and T2) for identifyingusers are composed in the entire image at the arbitrary resolution. Theuser identification images T are text-formed images each including aname of a user, for example, “trace of diagnosis of user A” or “trace ofdiagnosis of user B”.

The entire image 90 at the arbitrary resolution as described above isobtained by associating the history information items generated by thePC 100 through the observations for the users with the identificationinformation items for identifying the users and storing the associatedinformation items in the storage unit 108 or the like. That is, thehistory information 20 (see, FIG. 7) only has to be generated for eachuser. The text-formed images only has to be generated based oninformation on the user name of the PC 100 of each user or informationon a name that is manually input after the termination of theobservation of the images by the user.

The user identification images T are not limited to the text-formedimages. The user identification images T may be particular mark imagesthat are set for each user or arrow images that are set as imagesdifferent depending on the users. The number of users may be three ormore. The history images may be composed in different layers dependingon the identification information items. In this case, the PC 100 or thelike associates the identification information with the layerinformation to be stored in the storage unit 108 or the like.

Images in combination of at least two of the features in the entireimages at the arbitrary resolutions according to the first to fifthembodiments described above may be generated in the entire image at thearbitrary resolution.

Another Embodiment

The present disclosure is not limited to the above embodiments, andother various embodiments are conceivable.

In the above, the mode is described in which the image data that formsthe image pyramid structure 50 is stored in the storage unit 108 of thePC 100. However, instead of the PC 100, another computer or a server maystore the image data that forms the image pyramid structure 50, and thePC 100 used by the user as a terminal apparatus may access the computeror the server to receive the image data. In this case, the PC 100 as theterminal apparatus and the server or the like may be connected via anetwork such as a LAN and a WAN. In particular, the use of the WAN canrealize telepathology, telediagnosis, or the like.

In the above, the mode is described in which, as the original image ofthe image pyramid structure 50, one original image is generated withrespect to the one observation target object 15. However, with respectto the one observation target object 15, a plurality of original imagesmay be generated at different focus points in the thickness direction ofthe observation target object 15 which is a focus direction of theoptical microscope. This is called Z-stack, which is a function to dealwith the case where tissues or cells may have different shapes also inthe thickness direction of the observation target object 15. The scannerapparatus has the Z-stack function in many cases, and about 5 to 10 or10 to 30 original images are generated.

In addition, the scanner apparatus, the PC 100, or another computergenerates, from the plurality of original images thus generated, entireimages at resolutions for each original image (with respect to theplurality of original images) by a known compression method or a knowncompression method at a time when a thumbnail image is generated, forexample. That is, the entire images at a plurality of focus points aregenerated with respect to the entire image at least one resolution.Specifically, in this case, the total count of the entire images of theimage pyramid structure shown in FIG. 2 is determined to be M*L in whichM represents an existence count of the resolutions and L represents anexistence count of the focus points. L falls within the range of, forexample, 5 to 30 as described above, but is not limited to this range.Hereinafter, one of the plurality of images that are generated atdifferent focus points in the focus direction of the observation targetand that each include the plurality of original images will be referredto as a different-focus-point entire image.

In the case where such a different-focus-point entire image isgenerated, the PC 100 or another computer performs the followingprocessing, for example. That is, the PC 100 or the like only has toassociate positional information of different partial images as a partof the image in the different-focus-point entire image with focusinformation of those different partial images, and further store theassociation information as part of the history information 20. As aresult, for example, the PC 100 can generate a focus-point emphasizingimage that indicates the focus information in the entire image 40, 60,70, 80, or 90 at the arbitrary resolution. Consequently, the user canalso grasp the history on the focus points. The focus-point emphasizingimage is represented by at least one of a character, a sign, and afigure.

In the above description, the sampling period is set to be constant. Thesampling period in the case where the storing process is performed onthe history information of the partial image may be changed whennecessary by the user. Alternatively, the sampling period may becontrolled variably depending on the observation magnifications(resolutions). For example, the CPU 101 of the PC 100 sets the samplingperiod to a first time period in the case of the observation at a firstresolution, and sets the sampling period to a second time period that isshorter than the first time period in the case where a second resolutionthat is higher than the first resolution is set through a useroperation. As a result, the number of partial images at a highermagnification can be greater than the number of partial images at alower magnification.

Conversely, the CPU 101 of the PC 100 may set the sampling period to thefirst time period in the case of the observation at the firstresolution, and may set the sampling period to the second time periodthat is longer than the first time period in the case where the secondresolution that is higher than the first resolution is set through theuser operation.

The above-mentioned processing can allow the user to select importanceor priority with respect to the partial image of the high magnificationand the partial image of the low magnification.

Alternatively, the CPU 101 of the PC 100 counts a display time periodthat is a time period during which one partial image is displayed, andcompares the display time period with a certain sampling period. In thecase where the display time period is longer than the sampling period,the CPU 101 may store the display time period as the history informationof the one partial image. As a result, in the case where the displaytime period is shorter than the sampling period, the CPU 101 can omitprocessing of storing the history information of the partial image whosedisplay time period is short. Thus, the processing performed by the CPU101 becomes more efficient, resulting in reduction in quantity of memorythereof.

In the above embodiments, the image pyramid structure 50 is formed ofthe image group constituted of the entire images at the resolutions thatare generated and stored in advance. However, the PC 100, anothercomputer, or the like may generate, when necessary, an entire image thatis not generated and stored in advance. For example, the PC 100 or thelike can generate an entire image at a resolution corresponding to theobservation magnification of 30 times by interpolation from the entireimages whose observation magnifications of 20 times and 40 times whichare generated and stored in advance.

The PC is used as the information processing apparatus according to theabove embodiments, but a dedicated information processing apparatus maybe used instead of the PC. Further, the information processing apparatusis not limited to an apparatus that implements the above-describedinformation processing in cooperation with the hardware resource and thesoftware. Dedicated hardware may implement the above-describedinformation processing.

It should be understood that various changes and modifications to thepresently preferred embodiments described herein will be apparent tothose skilled in the art. Such changes and modifications can be madewithout departing from the spirit and scope and without diminishing itsintended advantages. It is therefore intended that such changes andmodifications be covered by the appended claims.

The application is claimed as follows:
 1. An information processingapparatus comprising: a processor; and a memory device operativelycoupled to the processor, the memory device storing instructions thatcause the processor, in cooperation with the memory device, to, for afirst observed image associated with an observation target object,associate and store first position information and first observationmagnification information.
 2. The information processing apparatus ofclaim 1, wherein the observation target object includes a section ofbiological tissue.
 3. The information processing apparatus of claim 1,wherein the instructions, when executed by the processor, cause theprocessor to, in response to a request for a change in observance fromthe first observed image to a second image to be observed, associate andstore the first position information and the first observationmagnification information.
 4. The information processing apparatus ofclaim 1, wherein the instructions, when executed by the processor, causethe processor to associate and store first position information andfirst observation magnification information based on a predeterminedsampling period.
 5. A method of operating an information processingapparatus including instructions, the method comprising: causing aprocessor to execute the instructions to, for a first observed imageassociated with an observation target object, associate and store firstposition information and first observation magnification information. 6.A computer-readable medium storing instructions structured to cause aninformation processing apparatus to: for a first observed imageassociated with an observation target object, associate and store firstposition information and first observation magnification information.